User engagement on
NGO Facebook pages

Effects of post sentiment measures over time

Nils Gustafsson & Nils Holmberg

Research questions

  • RQ1: How do Swedish nonprofits’ use of emotional expressions on Facebook change over time?
  • RQ2: What are the effects of emotional expressions on user engagement in Facebook posts from Swedish nonprofits?


  • ca 70 000 posts, after cleaning
  • from 125 pages (e.g. WHO)
  • time period of 2014-2019
  • downloaded using CrowdTangle

Engagement index: sum of likes, comments and shares

Sentiment dictionary

  • content coded for sentiment using the AFINN dictionary
  • better results with machine learning ?
  • quantitative content analysis vs computational

some caption

Sentiment measures

  • ca 70 000 posts from 125 organizational pages
  • time period of 2014-2019
  • text pre-processing, emojis converted to text descriptions
  • sentiment analysis on all text content


  • date of posting
  • number of followers
  • post language

some caption

Sentiment analysis

  • content coded for sentiment using the AFINN dictionary
  • words and emojis assigned valence value of -5 to +5
  • valence indicate positive or negative sentiment (0 being neutral)
  • valence squared indicates intensity of sentiment

some caption

Sentiment over time

  • sentiment normalized within organizations
  • normalized sentiment aggregated per month
  • positive relationship between sentiment and time?

some caption

Engagement and sentiment

  • engagement index: sum of likes, comments, shares

  • engagement index normalized within organizations

  • normalized engagment aggregated per month (mean)

  • positive relationship between engagment and sentiment?

some caption

Effects on engagement

  • positive effects
    • post sentiment valence
    • post sentiment intensity
    • number of followers at time of posting


  • negative effects
    • time, month number 1-60


  • type of organization?

Linear mixed effects model with organization as random factor

Variables by organization type

  • 1=culture and recreation
  • 2=education and research
  • 3=health
  • 4=social services
  • 5=environment
  • 6=development and housing
  • 7=law, advocacy an politics
  • 8=philantropic intermediaries and voluntarism promotion
  • 9=international

Engagement, mean over time

  • big differences between organizations
  • green: type health, large org
  • red: type international, small org

Engagement, trend over time

  • relatera sina akademiska
  • tillämpa teorier inom
  • utveckla erfarenheter att

Engagement measures

engagement index, dependent variable

    1. e_index_mean: per post e_index average per month (“raw engagement scores”)
    1. e_index_mean_norm: normalized per organization values of e_index_mean (1. above)
    1. e_index_norm_mean: per post e_index normalized per organization average per month
    1. e_index_rel_mean: per post e_index relative to follower count average per month

Engagement trend by organization type

  • relatera sina akademiska
  • tillämpa teorier inom
  • utveckla erfarenheter att

Engagement measures by organization type

  • relatera sina akademiska
  • tillämpa teorier inom
  • utveckla erfarenheter att

Thanks!